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Abstract 

Business Intelligence (BI) and Business Analytics (BA) Software has been included in many 
Information Systems (IS) curricula. This study surveyed current and past undergraduate and 
graduate students to evaluate various BI/BA tools. Specifically, this study compared several software 
tools from two of the major software providers in the BI/BA field. The participants in the study 
evaluated each software tool according to three key criteria: 1) functionality, 2) ease of use, and 3) 
learning effectiveness. The "learning effectiveness" criterion was used to determine which BI/BA tools 
provided the most effective learning of BI/BA concepts in the IS classroom. The three criteria were 
used to develop recommendations for including specific BI/BA software tools in the IS curriculum. 
Based on the findings of the study, the authors recommend that colleges and universities consider the 
use of the IBM-Cognos suite of tools as a viable means for teaching BI/BA concepts in their 
Information Systems curricula. The results of the study are relevant to any college or university that 
currently includes (or is considering the inclusion of) Business Intelligence / Business Analytics 
concepts in its Information Systems curriculum. 

Keywords: Business Intelligence, Business Analytics, Software Evaluation, Information Systems 
Curriculum 


1. INTRODUCTION 

Business Intelligence (BI) and Business Analytics 
(BA) Software has been included in many 
Information Systems (IS) curricula (Davis, 
Woratschek, & Kohun, 2005; Olsen & Bryant, 
2012). The authors of this study sought to 
determine which BI/BA software tools are the 
most effective in IS curricula. To determine 
software effectiveness, the authors surveyed 
current and past undergraduate and graduate 


students who are attending/have attended 
BI/BA-related courses. Specifically, the students 
who were surveyed were asked to compare 
several software tools from two of the major 
software providers in the BI/BA field. In order 
to determine a level of "effectiveness," the 
participants in the study evaluated each 
software tool according to three key criteria: 1) 
functionality, 2) ease of use, and 3) learning 
effectiveness. The "learning effectiveness" 
criterion was used to determine which BI/BA 
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tools provided the most effective learning of 
BI/BA concepts in the IS classroom. The three 
criteria were used to develop recommendations 
for including specific BI/BA software tools in the 
IS curriculum. The results of this study are 
relevant to any college or university that 
currently includes (or considers including) 
Business Intelligence / Business Analytics 
concepts in its Information Systems curriculum. 

2. RESEARCH QUESTIONS 

The current study attempted to determine the 
effectiveness of Business Intelligence (BI) / 
Business Analytics (BA) software, in regard to 
classroom use, by answering the following 
research questions: 

1. Which suite of BI/BA software tools (i.e., 
IBM-Cognos or Microsoft) was rated by 
survey participants as having greater 
functionality? 

2. Which suite of BI/BA software tools (i.e., 
IBM-Cognos or Microsoft) was rated by 
survey participants as having greater ease of 
use? 

3. Which suite of BI/BA software tools (i.e., 
IBM-Cognos or Microsoft) was rated by 
survey participants as providing greater 
learning effectiveness? 

4. If there are noted differences between IBM- 
Cognos and Microsoft BI/BA tool suites, are 
the differences statistically significant? 

3. BACKGROUND LITERATURE 

The term Business Intelligence (BI) was 
originally coined by Richard Millar Devens in 
1865. He used the term to describe how a 
banker profited by receiving and acting upon 
information about his environment before his 
competitors could (Devens, 1865). Collecting 
and acting upon information retrieved is still the 
basis of the definition of BI used today. 

A little less than a century later, the term BI was 
used by IBM researcher, Hans Peter Luhn. Luhn 
used Webster's dictionary definition of 
intelligence: "... the ability to apprehend the 
interrelationships of presented facts in such a 
way as to guide action towards a desired goal" 
(Luhn, 1958, p.314). 
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Business Intelligence, as the term is used today, 
evolved from the decision support systems 
(DSS) that began in the 1960s and developed 
throughout the mid-1980s. Modern BI systems 
only became a reality in the 1990s with the 
advent of the data warehouse. Many authors 
assert that Modern BI is not a technology. 
Rather, it is described as a process of generating 
information from raw data by using a 
combination of hardware, architectures, tools, 
methods, and databases (Turban, Sharda, 
Delen, & King, 2011). 

A review of the literature finds that many 
colleges/universities do not offer a degree in BI. 
A 2010 survey was conducted by the BI 
Congress to determine the state of BI in 
academia. This Congress is the work of the 
Teradata University Network (TUN) and the 
Special Interest Group on Decision Support, 
Knowledge and Data Management Systems 
(SIGDSS). Approximately 130 colleges/ 
universities were represented in this survey and 
173 professors responded. Only three schools 
reported having an undergraduate degree in BI: 
Augusta State University, St. Joseph's 
University, and Stuttgart Media University 
(Germany). 

Twelve schools reported having a graduate 
degree in BI: Augusta State University, 
University of Denver, St. Joseph's University, 
Stuttgart Media University (Germany), Sofia 
University (Bulgaria), North Carolina State 
University, Singapore Management University 
(Singapore), Texas Tech University, Loyola 
University Chicago, Xavier University, University 
of Muenster (Germany), and Universidade 
Portucalense (Portugal) (Wixom & Ariyachandra, 
2011 ). 

For those colleges/universities teaching BI 
courses, 34% indicated that having access to BI 
software was one of the challenges in teaching 
BI (Wixom, B. H. and T. Ariyachandra, 2011). 
Academic partnerships were used to access BI 
software/resources, specifically: Teradata 

University Network (48%) Microsoft Educational 
Consortium (46%), IBM Academic Alliance 
(28%), and Oracle Academy (14%) (Wixom & 
Ariyachandra, 2011). 

In 2012, the BI Congress once again surveyed 
colleges/universities to determine the state of BI 
in academia. Forty-three countries and 319 
professors were represented in the survey. The 
United States had the most respondents at 206 


©2015 EDSIG (Education Special Interest Group of the AITP) 
www.aitp-edsig.org /www.isedj.org 


Page 24 



Information Systems Education Journal (ISEDJ) 
ISSN: 1545-679X 


(66.5%). Germany came in second with 19 
respondents (6.1%). Only 26% of the 
respondents stated that one of the challenges in 
teaching BI was having "access to 
contemporary, enterprise software" (Wixom, 
Ariyachandra, & Mooney, 2013). The academic 
partnerships used to access BI 
software/resources were Microsoft Educational 
Consortium (46%), Teradata University Network 
(30%), IBM Academic Alliance (28%), and 
Oracle Academy (12%) (Wixom, Ariyachandra, & 
Mooney, 2013). 

Robert Morris University acquired a license for 
the academic use of Cognos' OLAP tool (i.e., 
PowerPlay) in 2003. However, that license 
expired and the software became unaffordable. 
Finding affordable BI software/resources for 
academic use was challenging. The solution was 
to join the Microsoft Educational Consortium, 
based out of the University of Arkansas, and use 
Microsoft's BI tools. The BI courses at Robert 
Morris University have used Microsoft's BI tools 
for the past three years. 

The large scale software vendors such as IBM, 
Oracle, SAP, Teradata, and Microsoft all boast of 
a BI solution to business problems. However, 
questions arise as to the affordability and vendor 
support of each of these solutions for collegiate 
classroom use. As equally important, what are 
the advantages/ disadvantages, from the 
standpoint of student learning outcomes, in 
using one vendor solution over another? 

4. METHODS AND PROCEDURES 
Approach and Sample 

This study involved the administration of a 
survey to current and past students who are 
currently taking (or have taken) Business 
Intelligence (BI) / Business Analytics (BA)- 
related courses. The survey population was 
obtained from student rosters of both 
undergraduate-level and graduate-level courses. 
The survey participants attended (or are 
currently attending) Robert Morris University, a 
private, medium-sized, Mid-Atlantic school. 
QuestionPro Online Survey Software, from 
QuestionPro, Inc. was used to administer the 
survey, collect results, and analyze results. All 
survey participants were over the age of 18, and 
participation in the study was completely 
voluntary. In addition, all survey responses 
were captured and stored anonymously (i.e., no 
personally-identifying information was solicited 
nor captured from the survey participants). 
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The QuestionPro online survey link was sent (via 
electronic mail) to 325 current and past 
students. The survey link was active and 
available from April 1, 2014 until April 30, 2014. 
During the 30-day period that the survey link 
was available, 46 respondents completed the 
survey and submitted their responses for 
analysis. The completion rate for the online 
survey was just over 14%. 

Survey Instrument 

The survey instrument consisted of a total of 27 
questions; 25 of the questions were closed- 
ended, and two of the questions were open- 
ended. The survey asked the participants to 
provide ratings for BI/BA software tools sold by 
IBM-Cognos and by Microsoft. These two 
software tools were chosen because of their use 
in Robert Morris University's BI courses, past or 
present. Questions one through nine asked 
participants to rate various BI/BA software tools 
provided by IBM-Cognos (i.e., Data Manager, 
Transformer, Analysis Studio, and Report 
Studio). Question 10 asked participants to give 
an overall rating to the suite of BI/BA tools sold 
by IBM-Cognos. Questions 11 through 19 asked 
participants to rate various BI/BA software tools 
provided by Microsoft (i.e., Integration Services, 
Analysis Services, Excel, and Reporting 
Services). Question 20 asked participants to 
give an overall rating to the suite of BI/BA tools 
sold by Microsoft. In all of the questions that 
solicited a rating, participants were asked to rate 
the tools according to functionality, ease of use, 
and learning effectiveness. For each of the 
aforementioned criterion, participants were 
asked to provide a rating of (1) Poor, (2) 
Average, (3) Good, or (4) Excellent. The four- 
point rating system was used in the survey in 
order to require a "forced-choice" from the 
participants and, therefore, avoid "central 
tendency" bias. 

In addition to the questions that solicited a 
rating, the survey also contained several 
demographic questions. The demographic 
questions asked participants to indicate their 
degree (i.e., either earned or in-progress), their 
sex, whether or not they are currently working 
in BI/BA, and (if "yes") what BI/BA tool(s) they 
currently use in their job. 

Toward the end of the survey, participants were 
asked what they felt would be the next "hot 
topics" in the field of BI/BA. At the very end of 
the survey, participants were asked to list the 
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BI/BA topics that they feel should be included in 
Information Systems curricula. 

Once collected, all survey results were analyzed 
using SPSS (Statistical Package for the Social 
Sciences). Descriptive statistics were generated 
in SPSS to calculate the participants' mean 
rating scores, as related to software 
functionality, ease of use, and learning 
effectiveness. In addition, the Independent 
Samples T-Test was used to determine if any 
noted differences in mean rating scores between 
the two software vendors were statistically 
significant. 

5. RESULTS 

Functionality 

To address the first research question (which 
Business Intelligence/Business Analytics tool 
suite was rated by participants as having greater 
functionality?), the survey contained questions 
that asked participants to compare the 
functionality of IBM-Cognos BI /BA tools with the 
functionality of Microsoft BI/BA tools. The 
functionality was categorized by ETL (Extract, 
Transform, and Load) functionality, OLAP (On¬ 
line Analytical Processing) functionality, and 
Reporting/BPM (Business Performance 
Management) functionality. Overall, the survey 
participants rated the functionality of IBM- 
Cognos BI/BA tools as being greater (f = 3.12) 
than the Microsoft BI/BA tools (f = 2.90). 

The Independent Samples T-Test was used to 
determine whether or not the difference in 
functionality was statistically significant. 
Although the IBM-Cognos tools were rated as 
having greater functionality than the Microsoft 
tools, the difference in means was not 
statistically significant at the .05 level (t = 
1.013, p = .316). The results from the 
responses regarding functionality are 
summarized in Appendix A - Table 1: 
Independent Samples T-Test for BI/BA 
Functionality. 

Ease of Use 

To address the second research question (which 
BI/BA tool suite was rated by participants as 
having greater usability?), the survey contained 
questions that asked participants to compare the 
usability of IBM-Cognos BI /BA tools with the 
usability of Microsoft BI/BA tools. In a similar 
manner as functionality, usability was again 
categorized according to the usability of ETL 
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tools, OLAP tools, and Reporting/BPM tools. 
Overall, the survey participants rated the 
usability of IBM-Cognos BI/BA tools as being 
greater (f = 3.00) than that of Microsoft BI/BA 
tools {x = 2.62). 

The Independent Samples T-Test was used to 
determine whether or not the difference in 
usability was statistically significant. The IBM- 
Cognos tools were rated as having greater 
usability than the Microsoft tools; however, the 
difference in means was not statistically 
significant at the .05 level (t = 1.653, p = .105). 
The results from the responses regarding 
usability are summarized in Appendix A - Table 
2: Independent Samples T-Test for Ease of Use. 

Learning Effectiveness 

To address the third research question (which 
BI/BA tool suite was rated by participants as 
having greater learning effectiveness?), the 
survey contained questions that asked 
participants to compare the learning 
effectiveness of IBM-Cognos BI /BA tools with 
the learning effectiveness of Microsoft BI/BA 
tools. As with the prior criteria, learning 
effectiveness was categorized according to the 
learning effectiveness of ETL tools, OLAP tools, 
and Reporting/BPM tools. Once again, the 
survey participants rated the IBM-Cognos tools 
higher than Microsoft. Overall, the participants 
rated the learning effectiveness of IBM-Cognos 
BI/BA tools as being greater (x = 3.18) than the 
Microsoft BI/BA tools (f = 2.86). 

The Independent Samples T-Test was used to 
determine whether or not the difference in 
learning effectiveness was statistically 
significant. The IBM-Cognos tools were also 
rated as having greater learning effectiveness 
than the Microsoft tools; however, the difference 
in means was not statistically significant at the 
.05 level (t = 1.711, p = .094). The results 
from the responses regarding usability are 
summarized in Appendix A - Table 2: 
Independent Samples T-Test for Learning 
Effectiveness. 

T-Test for Statistical Significance 

As discussed in the METHODS AND 
PROCEDURES section, the Independent 
Samples T-Test was used to detect statistical 
significance in any noted difference in survey 
results between the two vendors. None of the 
criteria tested (i.e., functionality, ease of use, 
nor learning effectiveness) differed between the 
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two software vendors in a statistically significant 
way. Out of all three criteria tested, the 
learning effectiveness criterion came closest to a 
statistically significant difference. However, as 
discussed above, the difference in means 
between IBM-Cognos BI/BA tools and Microsoft 
BI/BA tools for learning effectiveness was not 
statistically significant at the .05 threshold (t = 
1.711, p = .094). 

6. CONCLUSIONS 

This research surveyed current and past 
undergraduate and graduate students to answer 
the following questions regarding IBM-Cognos 
and Microsoft BI/BA (Business Intelligence / 
Business Analytics) software tools: 1) Which 
suite of software tools was rated by survey 
participants as having greater functionality?, 2) 
Which suite of software tools was rated by 
survey participants as having greater ease of 
use?, 3) Which suite of software tools was rated 
by survey participants as providing greater 
learning effectiveness?, and 4) Are any noted 
differences in participant ratings between the 
two software vendors statistically significant? 

As discussed in the RESULTS section, the IBM- 
Cognos tools were rated higher than Microsoft 
tools by survey participants in all three 
categories: 1) functionality, 2) ease of use, and 
3) learning effectiveness. However, none of the 
differences were statistically significant at the 
.05 level. Despite the lack of statistical 
significance, these results seem to indicate that, 
when compared to Microsoft, the IBM-Cognos 
BI/BA suite of tools offer greater functionality for 
performing BI/BA tasks, and greater ease of use 
for the end user. These results also seem to 
indicate that the IBM-Cognos tools provide more 
effective learning of BI/BA concepts (when used 
in the classroom) than the Microsoft toolset. 

The above findings are interesting when viewed 
in the context of responses received from other 
survey questions. For example, more survey 
participants (61%) reported as having used the 
Microsoft BI/BA toolset than the IBM-Cognos 
toolset (39%). This pattern of software use is 
not surprising, given that Robert Morris 
University has been using the Microsoft BI/BA 
toolset since 2011. In other words, it is 
expected that current and past students of the 
University would have had exposure to the 
Microsoft toolset, as part of their BI/BA classes. 
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The findings, regarding the use of BI/BA 
software in academia, are also consistent with 
the aforementioned findings by the BI Congress. 
As discussed previously, a 2010 survey by the BI 
Congress found that 46% of schools surveyed 
utilize the Microsoft Educational Consortium to 
provide BI/BA software to students. In the 2010 
survey, the percentage of schools utilizing the 
Microsoft Educational Consortium was 
significantly higher than the percentage of 
schools utilizing the IBM Academic Alliance 
(28%) (Wixom & Ariyachandra, 2011). 

However, when the current study asked which 
suite of tools was used in the workplace (by 
participants who currently work in the BI/BA 
field), IBM-Cognos was reported as the tool of 
choice for 21% of the participants' employers. 
The Microsoft BI/BA toolset, however, was 
reported as being used by only 13% of 
employers. It is also interesting to note that 
Oracle BI/BA tools were also reported as being 
used by 21% of employers. The other major 
BI/BA software vendors used by participants' 
employers included SAP-Business Objects 
(13%), and Informatica (15%). Oracle, SAP- 
Business Objects, and Informatica software tools 
were not rated by participants in the current 
study. The remaining 17% of employers in 
question were either represented by the 
category "Other" (i.e., we use a BI/BA tool that 
was not listed on the survey) or by "My 
organization does not use BI/BA software tools." 

Recommendations 

Based on the findings from the current research, 
colleges and universities should consider the use 
of the IBM-Cognos suite of tools as a viable 
means to for teaching BI/BA concepts in their 
Information Systems curricula. However, these 
findings, and the resulting recommendations, 
are contrary to the reported market shares of 
leading BI/BA software suites. In North 
America, the top three BI/BA vendors, in terms 
of 2013 market share, were Microsoft (43%), 
Oracle (30%), and SAP-Business Objects (28%) 
(Henschen, 2014). In terms of 2013 worldwide 
market share, the top three BI/BA vendors were 
SAP-Business Objects (21%), Oracle (14%), and 
IBM-Cognos (13%) (Columbus, 2013). 

The recommendations from the current study 
also seem to conflict with the current use of 
academic partnerships by colleges and 
universities. As discussed previously, the most 
prevalent academic partnerships leveraged by 
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colleges and universities are Microsoft 
Educational Consortium (46%), Teradata 
University Network (30%), IBM Academic 
Alliance (28%), and Oracle Academy (12%) 
(Wixom, Ariyachandra, & Mooney, 2013). 

Future Research 

Even though the IBM-Cognos suite of BI/BA 
tools received the highest rating in all three 
question categories, the sample size of the 
current study was quite limited (n = 46). Future 
research could solicit responses from a larger 
sample, both in terms of number of participants 
and in geographical area. Participant ratings for 
additional BI/BA software vendors (e.g., Oracle, 
SAP-Business Objects, Teradata, Informatica, et 
al.) might also be requested. Finally, the 
current research focused on suites of BI/BA 
tools. Future research could solicit and analyze 
participant ratings for suites of tools, as well as 
individual types of tools, such as ETL, OLAP, 
Reporting, and BPM. 
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APPENDIX A - T-TEST RESULTS 


Table 1: Independent Samples T-Test Results 

Independent Samples T-Test Results for BI/BA Functionality 



Mean 

Std. t-test 

Dev. 

df 

Sig. 

IBM-Cognos Suite of 
BI/BA Software 

3.12 

.781 1.013 

44 

.316 

Microsoft Suite of 

BI/BA Software 

2.90 

.673 



Table 2: Independent Samples T-Test Results 

Independent Samples T-Test Results for Ease of Use 


Mean 

Std. Dev. t-test 

df 

Sig. 

IBM-Cognos Suite of 
BI/BA Software 

3.00 

.707 1.653 

44 

.105 

Microsoft Suite of 

BI/BA Software 

2.62 

.775 



Table 3: Independent Samples T-Test Results 

Independent Samples T-Test Results for Learning Effectiveness 


Mean 

Std. Dev. t-test 

df 

Sig. 

IBM-Cognos Suite of 
BI/BA Software 

3.18 

.636 1.711 

44 

.094 

Microsoft Suite of 

BI/BA Software 

2.86 

.581 
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